On a multiple nodes fault tolerant training for RBF: Objective function, sensitivity analysis and relation to generalization

Over a decades, although various techniques have been proposed to improve the training of a neural network to against node fault, there is still a lacking of (i) a simple objective function to formalize multiple nodes fault and not much work has been done on understanding of the relation between fault tolerant and generalization. In this paper, an objective function based on the idea of Kullback-Leibler divergence is presented for multiple nodes fault tolerant training. It is essentially the same as a summation of mean square errors plus a regularizer. A simple training algorithm for attaining fault tolerant neural network is presented accordingly and its gracefully performance degradation is shown by simulation results. Besides, the sensitivity of the training algorithm against node fault rate is analyzed and its insensitivity is demonstrated by simulation results. Finally, a discussion on fault tolerant and generalization is presented and the incapability of using regularizers for improving generalization to achieve optimal fault tolerant is commented.